Integrated CMOS imaging array and dark current monitor 



1 FIELD OF THE INVENTION 

2 The field of the invention is the field of digital imaging using complementary metal 

3 oxide semiconductor (CMOS) technology. 

4 BACKGROUND OF THE INVENTION 

5 Modem imaging systems using charged coupled device (CCD) arrays and 



6 O complementary metal oxide semiconductor CMOS arrays have come into increasing use for both 

7 Q industrial and consumer applications. CCD arrays have a longer history and much more time to 

8 m solve problems arising m manufacture and use. CCD arrays have fewer dark current problems 

9 than CMOS arrays. However, CMOS arrays promise less expensive imaging, principally 

10 because other functions may be combined on the semiconductor substrate and the well honed 

1 1 m techniques developed for computer technology may be used to produce the arrays. 

12 05 Prior art CMOS arrays have an array of pixels arranged in rows and cohimns, where light 

O 

13 m is absorbed in the semiconductor substrate to generate electrons, and the electrons are stored until 

14 they are "read out" to generate an image. The charge stored for one pixel is generally sent 

1 5 through a column amplifier to generate a voltage, which is then converted to a digital signal in an 

16 analog to digital converter (ADC). The ADC has a minimum signal input V^,„ below which a 

1 7 zero is generated, and a maximum signal input V^^^ which saturates the ADC. Each column 

1 8 amplifier generally has an offset voltage which is added to the signal fi*om the stored electrons, 

19 so that the signal sent to the ADC is W^^,^ if no light is incident on the pixel and no electrons are 

20 stored. Since the amplification is slightly temperature and process dependent, the offset voltage 

21 must be adjustable, and means provided to send a "zero" or "dark" signal to the column amplifier 

22 so that the offset may be set to a voltage slightly less than V^,„. The "dark" signal may be sent 

23 by dummy pixels which in effect ground the input to the amplifier. 



Docket No 901-010 Inventor Adam Jacobs -Express Mail EI531 153305US 
Integrated CMOS imaging array and dark current monitor- deposited February 8, 2002 



1 



1 However, electrons are generated in the pixel without light by spontaneous means which 

2 are generally exponentially dependent on the temperature of the semiconductor material of the 

3 pixel The electrons so generated give rise to "dark current". Most light sensors produce "dark 

4 current". The prior art of CMOS image arrays is deficient in that the "dark current" is not 

5 accoxmted for in the case that the array temperature, and also the variation in temperature over the 

6 array, changes with time. For the case of a CMOS integrated circuit application, where many 

7 functions are carried out on a monolithic semiconductor chip, this is a serious deficiency. Each 

8 part of the chip produces a different amount of heat at different times. Part of the array may be 

9 near a section of the chip having a dense array of devices liberating a large amount of heat, so 

10 u that parts of the array vdll have a large temperature difference with respect to other parts of the 

1 1 if array. The temperature and variation in temperature may change as the device is used for 

12 '/^ imaging, so that the dark current contribution to the image is variable and unaccounted for. 

13 Pi OBJECTS OF THE INVENTION 

14 1^;: It is an object of the invention to provide an apparatus and a method to account for the 

1 5 2 contribution of dark current to the output of a CMOS imaging array. 

16 fll It is an object of the invention to provide an apparatus and a method to account for the 

1 7 contribution of changing temperature to the contribution of dark current to the output of a 

1 8 CMOS imaging array. 

19 It is an object of the invention to provide an apparatus and a method to account for the 

20 contribution of temperature variation over a chip to the contribution of dark current to the output 

21 of a CMOS imaging array. 

22 It is an object of the invention to provide a device to monitor dark current of a CMOS 

23 imaging array. 

24 It is an object of the invention to provide a device to monitor temperature over a CMOS 

25 imaging array and calculate dark current contributions to the output of the CMOS imaging array. 

26 SUMMARY OF THE INVENTION 
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1 The present invention is a system, apparatus and method for accounting for the dark 

2 current provided in each pixel of a CCD or a CMOS imaging array during the exposure of the 

3 array to light in an imaging process. The dark current from permanently darkened pixels, or the 

4 temperature at one or more points on the semiconductor chip containing the array, is monitored 

5 during the imaging process. The actual dark current measured from the darkened pixels, or a 

6 dark current calculated from the temperature, is used in conjunction with a lookup table of dark 

7 currents previously measured to calculate the contribution to the signal from each pixel expected 

8 from the dark current generated in each pixel during the exposure. Both the temperature 

9 dependent dark current and the fixed pattem dark current contribution is then subtracted from 
lot! the signal measured from each pixel, and the corrected image may then be displayed or used for 

11 O further manipulation or storage. 

ss i; 

m 

1 2 BRIEF DESCRIPTION OF THE DRAWINGS 

1 3 O Fig. 1 shows a block diagram of the method of the invention. 

14 u Fig. 2 shows a sketch of the apparatus of the invention. 

15 Fig. 3 shows a sketch of the system of the invention. 
16 

17 DETAILED DESCRIPTION OF THE INVENTION 

1 8 To achieve repeatability of performance of an imaging system, it is necessary to 

1 9 compensate for or remove sensor and illumination artifacts in the system. In critical 

20 applications, such as imaging of biological tissue for diagnosis of such diseases as cancer, the 

21 performance of the imaging system is critical. The imaging system is made up of an 

22 illumination system for illuminating an object, an optical system for projecting an image of the 

23 object on to an image detector, and hardware and software for producing a computer and/or 

24 human readable output for further evaluation. Our purpose in imaging an object is to quantify 

25 the amount of light emitted at each point on the target, so as to most accurately map the radiant 

26 emittance from each point on the object. For purposes of the following discussion, we make the 
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1 simplifying assumption, which need not be true, that transillimiination and multiple-scattering 

2 effects are sufficiently small that all of the light emitted can be considered to arise from 

3 reflectance. As an example, a CMOS technology imaging array will be considered. Such a 

4 CMOS array is generally constructed on a single chip of silicon, but may in fact be constructed 

5 on a chip of any semiconductor material, including but not limited to silicon and silicon- 

6 germanium, GaAs, and other near and far Infra-red unaging material. Other imaging array 

7 techniques such as CCD arrays as known in the art or to be invented in future are also anticipated 

8 by the inventors. 



9 sssji: 


Known random noise sources include: 


10 2 • 


Photon Shot Noise 


11 nJ • 


Pixel kTC{ capacitive) Noise 


12 M • 


Dark Current Shot noise 




Pixel source follower noise 


14 • 


Column amplifier source follower noise 


15 PJ • 


Vertical sample and hold noise 


16 ft • 


Colunm amplifier transistor noise 


17 • 


kT C noise on double sampling amplifier noise 


18 • 


Horizontal sample and hold 


19 • 


Output amplifier noise 



20 Each of these types of noise is random, and cannot be removed by calibration. 

21 However, effects that are fixed (i.e., "deterministic") can be removed. These include: 

22 • dark current non-uniformity 

23 • amplifier offset variations 

24 • pixel quantum efficiency (QE) variation 

25 • amplifier gain variation 
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1 • illumination variation 

2 The number of photons reaching the f pixel of the sensor from a particular spot on the 

3 target corresponding to the pixel is proportional to the illumination {I) at that spot, multiplied 

4 by the (bidirectional) reflectance (R,) of the same spot. (This "local" relation assumes that there 

5 are no significant transillumination or lens effects.) The photons reflected from the spot and 

6 received at the imaging array are converted to electrons at the /'^ pixel of the sensor with 

7 quantijm efficiency QE,. 

8 The number of electrons at each pixel in the sensor is converted to a voltage V, that is 

9 , , ultimately measured, i.e., the number of electrons at conversion time is what is output from the 

10 y sensor. 

1 1 H However, electrons are created within the sensor pixel sites through two mechanisms. 

12^1 • dark current 

13 • photons converted to electrons 

14 IB The number of electrons arising from dark current in the f pixel (k^,) is generally 

1 5 n J proportional to exposure time ( r) and exponentially dependent on temperature (7), which we 

16 express in functional form as follows: 

17 (1) n,,=nLT) 

1 8 The number of photons , converted to electrons is given by the number of photons 

1 9 reaching the sensor times the quantum efficiency of that pixel and is expressed by the 

20 proportionality expression: 

21 (2) n^^=kJ,R,QE, 
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Therefore the total number of electrons n , in the pixel after the exposure is given by the number 
fi"omthe Dark Current + the number generated by photons 



3 (3) n,= n,,^ n^, ^ f^T^r) J ,R , QE, 

4 These electrons are passed to one or more amplifiers which have a cumulative gain (G,) and an 

5 offset (Oi) for each pixel. Thus, the signal {S, ) at the output of the amplifier (which is the output 

6 of the sensor) can be expressed as follows: 

lu (4) S,=GX fkT.r)^kJ,R,QE,)W,, 

8 J':^ where / = pixel index. We can invert this equation and solve for the unknown reflectance R , of 

9 nJ the target (e.g., skui). There are variations pixel to pixel in G,, / (7^ ^ , A: , , / , , QE,2xA0,, 

10 S| We are now left with the task of determining these quantities pixel to pixel to solve for the 

1 1 □ reflectance at that pixel. 

12 S In the most preferred embodiment of the invention some of the pixels of the array are 

1 3 i2 permanently covered or blacked over so that no light reaches them. The only signal that comes 

14 fll from these darkened pixels is due to the dark current, multiplied by the pixel amplifier gains, and 

1 5 a fixed dark pattem offset. As the temperature and exposure time of the sensor array changes, 

16 the charge measured from these darkened pixels will predict the change in the dark current 

17 contribution of the neighboring pixels which are exposed to light. In another preferred 

18 embodiment, one or more darkened extra pixels, arrays of pixels, or discrete light detection 

1 9 elements are placed in areas of the chip outside or inside the array. In another preferred 

20 embodiment, the temperature of the array is monitored by one or more temperature monitors, 

2 1 relative to a temperature when the dark current of the entire array has been measured, and the 

22 relative dark current is calculated from the monitored temperature by external circuitry or by 

23 circuitry integrated on the same monolithic substrate as the image array. In preferred 

24 embodiments, the dark current or temperature variations across the array is calculated by 
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1 extrapolation and/or interpolation of the dark current or temperature monitor results at specific 

2 points inside or outside the array. 

3 At a constant current bias, the voltage drop across a silicon P-N diode junction shows 

4 roughly a ~2 mV/ "^C temperature coefficient. Relative temperature measurements may be made 

5 very inexpensively by measuring such a voltage drop. More accurate temperature measurements 

6 my be made by a large variety of PTAT (proportional to absolute temperature) circuits which are 

7 very well known in the art. Examples of such circuits are given in United States Patent 

8 5 ,7985669, which is incorporated herem by reference ♦ " 

9 Fig. 1 shows a block diagram of a preferred method of the invention. In block 10, a 

1 0 CMOS imaging array is provided with a block to prevent any light from reaching at least one of 

11 y the pixels of the array. The blocked or darkened pixels are used to monitor the dark current in 

12 y the lighted pixels. In practice, many pixels must be blocked or darkened because the noise in 

1 3 flJ the measurement of just one pixel would be too great for accurate measurement of dark current. 

1 4 %i The blocked or darkened pixels are preferably in one comer of the CMOS array. Another 

15 y preferable position for the blocked pixels is in each of the 4 comers of the (generally rectangular) 

16 Jif arrays. In another preferred embodiment, the pixels are on one or more edges, or in any 

1 7 " convenient place on the array, depending on the expected characteristics of the images. 

18 flJ In block 12, a very short exposure is taken. The exposure is so short that r is small so 

19 that the terms including and n^^ is negligible in equation 4, and =S,, We assume 

20 that the offset changes Uttle with temperature T and with exposure time x, wavelength X, and 

21 exposure intensity. The short exposure may be made at any time durmg the calibration of the 

22 imaging array to check that the offset has not drifted since the last time it was measured. The 

23 signals S, measured in block 12 are recorded as the offsets O , for each pixel 

24 In block 14, the entire array is kept in the dark, and the signal output from the array is 

25 measured for a long time exposure. In this case, the illumination / ^ is zero, and the contribution 

26 to the signal due to the dark current ( /(J, r)) is determined from the results of step 14 

27 and the offsets measured in step 12. Multiple exposures with different times t are preferably 

28 used to check that ( /(r, r)) is approximately linear in r. The dark current measured from 
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1 the normally lighted pixels in block 14 is recorded, preferably in a look up table, as well as the 

2 dark current from the darkened pixels. In later exposures, the contribution of the dark current 

3 to the lighted pixels may be found from the values in the look up table using signal values 

4 measured from the darkened pixels when an image is taken. The dark current contribution to 

5 the illuminated pixels is proportional to the dark current contribution measured from the 

6 darkened pixels. If the temperature of the array changes, and the dark current in both the lighted 

7 and unlighted pixels changes exponentially with respect to the temperature. 

8 In a normal setting, it is not convenient to block the light from the sensor (as with a 

9 shutter in a camera) so that the dark current lookup table is preferably constructed in the factory 

1 0 C before the imaging system is shipped. However, measuring the dark current by blocking the 

1 1 H light from impinging on the imaging array may be carried out in the field to check that the 

12 if characteristics of the array have not changed. In an altemative preferred embodiment, the dark 

13 current contribution from the unilluminated array is measured and recorded and the same time as 

14 J' the relative or absolute temperature of the array is measured. When an image is recorded with 

1 5 Sf : the array illuminated, the temperature of the array is also monitored. The contribution of the 

16 g dark current to the image signal is then calculated by assuming the dark current contribution 

17 O varies exponentially with the temperature of the array. Such calculations are easily carried out by 

18 circuitry on the array chip, or by associated circuitry or computers off the chip. 

1 9 In block 1 6, a uniformly reflecting target is illuminated with high enough illumination for 

20 a short enough time that the dark current signal is negligible with respect to the photon generated 

21 signal, and 

22 (5) =G,(kJ,R, QE,) +0, 

23 Subtracting the measured values of , which are relatively independent of the illumination 

24 level, exposure time, and wavelength X of the illuminating light, and assuming that R ji is 

25 constant over all the pixels for each wavelength X, we can sweep all the variations in 

26 illumination, quantum efficiency, and gain into one measured ejffective gain coefficient 

27 (6) G* = G.ikJ, QEJ 

28 so that for an image of the target taken at the same illumination, wavelength, and exposure time 

29 as block 14. 
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1 (J) s,-g:^r,^ o,^Gjp:,T) 

2 and the reflectivity R ^ may be determined from the measured values of G * , the offset 

3 measured in block 12, and the dark current contribution f{T, t) determined from the 

4 measurements of block 14 and measurements of the signal from the darkened pixels . For 

5 differing illummation intensity, wavelength, and exposure time, the term muhiplying R , is 

6 calculated from the measured G * to recover R , . 

7 Fig. 2 shows a sketch of a monolithic semiconductor chip 20 with a rectangular imaging 

8 array 22. Portions 24 of the imaging array are shown having the light blocked. The four comers 

9 of the array are shown blocked, but only one portion of the array may be blocked, and it may be 

10 in any position in the array 22. Separate arrays or detectors on the chip 20 may also be used. 

1 1 O Portions 26 of the chip 20 preferably but not necessarily have devices such as amplifiers, analog 

12 L,Z to digital converters, memory and logic necessary to capture and image, translate the image from 

13 ^ analog to digital form and to carry out manipulations on the digital image to remove the effects 

1 4 of dark current on the digital image. In the most preferred embodiment, such manipulations are 

1 5 O carried out on the monolithic semiconductor chip. In other preferred embodiments, such 

16 O manipulations may be carried out off chip or in a separate computer. Regions 24 also represent 

1 7 p temperature monitoring devices for monitoring the temperature of the array 22. Such 

1 8 ^ ^ temperature monitoring devices may be located on the chip 20 inside and/or outside the array 22 . 

19 Fig, 3 shows a sketch of a system for using the apparatus and method of the invention. 

20 The object 30 is imaged on the imaging array 22 of chip 20 by an optical system exemplified by 

21 lens 32. Illumination is provided by illumination sources 34 which are preferably lamps and 

22 most preferably light emitting diodes (LED's) or laser diodes. Circuitry 38 controls the 

23 illumination sources 34 and the imaging array and electronic circuitry on chip 20. Circuitry 38 

24 receives data from chip 20, and optionally displays results on a monitor 37 and optionally stores 

25 results on a storage device 39, which may be located near the optical system 32 or remotely 

26 located and reached by intemet or other hard wire or software connection. Circuitiy 38 may be 

27 located on chip 20, near chip 20 on a circuit board, or may be a separate computer. 

28 Experimental Setup: 
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1 We used a normal Melafind^^^ 1 00 camera from Electro-Optical Sciences, Irvington, NY, 

2 which illuminates with a specific pattern of light for each wavelength, generally using different 

3 patterns for LED's of different colors. Images were taken with the imaging sensor in the dark 

4 for 1, 50, 100, 150, 200, 250 and 300 milliseconds. Then, images were taken of a photographic 

5 paper target that had been exposed to a uniform flash density and developed, with the same 

6 exposure times for each wavelength of illumination. 

7 The images were calibrated using the method of Fig. 1 , for a sensor array which had a 

8 number of pixels in one comer blocked by aluminum foil. 

9 Measurements were made after the chip had been run for a few minutes, and had a 

1 0 M temperature estimated to be 50**C. The signal to noise ratio measured from test exposures 

1 1 S increased from 15:1 with no correction for dark current to 200:1 with a correction for dark 

12 y current by the method of Fig. 1. 

13 ^ Obviously, many modifications and variations of the present invention are possible in 

14 H light of the above teachings. It is therefore to be understood that, within the scope of the 

1 5 p appended claims, the invention may be practiced otherwise than as specifically described. 
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